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There are basically four dimensions to a systems analysis that we find in current 
educational research: (1) Collecting information. (2) analyzing input-output 
relationships. (3) deriving models built upon these relationships to evaluate 
alternatives and to derive feasible solutions, and (4) attempting to derive the best 
solution consistent with the constraints. Three studies presently under way may help 
to provide insights into the range of educational research problems which might be 
beneficially examined from the systems analytic viewpoint. The first study deals with 
collecting information for building an evaluation system for elementary schools. The 
second is an attempt to develop a series of regression models for simulating the 
characteristics of secondary schools. The last deals with optimization of salary 
schemes. (DE) 
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Systems Analysis Applications in 
Educational Research 

Marvin C. Alkin 
Rodney W. Skager 

"Systems" is an amorphous word; its meaning depends a great deal 
upon who uses it. Some speak of philosophical systems while others are 
concerned with control systems, and still others with political systems, 
social systems or weapons systems. Some use systems to identify a 
physical construct while others use it to. describe a conceptual approach. 
Many use the term to indicate their concern with complicated organiza- 
tions made up of many interrelated parts. Each has equal claim to the 
word. We define a system as a set of interrelated factors (inputs) 
that are used together to produce at least one output. 

Systems are often in whole or in part enveloped by larger systems. 
For example, a classroom, a school, a district, a state's or nation's 
educational facilities can be legitimately defined as systems. The 
delineation of a specific system depends upon the decisions one wishes 
to make and the related questions the analyst wishes to answer. The 
generic question the systems analyst attempts to answer is how can the 
systems output be maximized in the direction of the desired objectives, 
utilizing available resources. To this end he must evaluate the resource 
cost and the corresponding outputs associated with combinations of 
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inputs. In essence, systems analysis is the comparison of alternative 
processes for achieving an objective when these processes are fairly 
complex and there are a number of interrelated elements. 

The paper presented by Robert Morgan has discussed one applica- 
tion of systems analysis. In that instance, systems analytic tech- 
niques were utilized in planning an educational program. There are a 
number of other examples of systems analysis applications to education 
that might be cited. Many of these are referred to in the book. 

New Look at Education , sponsored by E.T.S. and written by John Pfeiffer. 
It seems hardly necessary to repeat a description of the applications 
discussed in this extensive report. 

In addressing ourselves to the topic of this paper we found 
considerable difficulty in distinguishing between systems analysis 
application to education as exemplified by the E.S. 70 project and 
systems analysis application in educational research. 

In some respects the discussion of a topic such as this implies 
a lack of such applications in educational research. That is not the 
case; there are a number of applications of systems analysis to educa- 
tional research and most are worthy of and would require considerable 
time to discuss in detail. Moreover, many research, data gathering, 
and educational decision making situations presently taking place are, 
in effect, systems analyses. 

Since it is not possible to consider all of these applications, 
we will limit our discussion to several studies presently being 



erlc'7 









mdessasm 



mm 



















3 



performed at UCLA, and more specifically at the Center for the Study 
of Evaluation at that institution. One of the major research thrusts 
of the Center is concerned with evaluating educational systems. In 
these activities the Center has maintained that there is a necessity 
for considering a broad range of criterion, contextual, and instruc- 
tional variables, for understanding the nature of the complex inter- 
actions between these variables, and for performing such evaluation 
research within naturalistic school settings in order to have it be 
of real value to education. This systems analytic viewpoint of the 
Center provides the framework for several studies which will be dis- 
cussed briefly here. 

There are basically four dimensions to a systems analysis that 
we find in current educational research. These might be classified 
as (1) collecting information; (2) analyzing input-output relationships; 
(3) deriving models built upon these relationships in order to evaluate 
alternatives and to derive feasible solutions, and finally, (4) 
optimization where the research attempts to derive the best solution 

consistent with the constraints. 

Three studies presently underway at UCLA fall into these cate- 
gories. The first study we will discuss deals with the collecting 
of information which eventually would be used for educational decision 
making. The second project encompasses two of the functions previously 
defined. Present efforts within that study entail the analysis of 
relationships between inputs and outputs, with future stages being 
concerned with the evaluation of alternatives and the selection of 
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feasible solutions. The third study to be discussed is an optimi- 
zation problem. It has as its concern the selection of a single 
best solution from a host of feasible solutions. 

One project within the Center is concerned with building an 
evaluation system for elementary schools. The project has been 
planned as a 5-year development program encompassing systems analysis 
procedures. During this period the major stages of systems analysis 
will have been applied including the definition of objectives of the 
system, analysis of requirements and formulation of system specifi- 
cations, identification of major subsystems including analysis of 
interfaces or interrelations among subsystems, a comprehensive 
design phase, a thorough and intensive test phase including both 
simulated and actual tests and a final modification phase to pro- 
vide an effective system for operational use with review and feed- 
back continuing through the early operational use period. 

To elaborate further on this application, some of these major 
stages relative to the development of an elementary school evaluation 
system may be described in somewhat more detail. Of major importance 
in system analysis is the clear definition of the system's objectives 
and insuring that these objectives are ever-present and considered 
throughout all subsequent stages. The general objective for the ESES 
program was stated as a dual objective as follows: 

1. To develop a complete evaluation system that is accurate, 
sensitive, reliable and cost-effective. This will include 
a management information system that contains information 
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necessary for decisions relative to a. single school and 
its programs and emphases. Included also will be individ- 
ual school diagnostic information for improving perfor- 
mance of its grade levels, classes or other groups of 
individuals. 

2. To provide data leading to new information and concepts 
relative to evaluation of a school program, including the 
identification of important variables and their interrela- 
tionships and the exploration of cause-effect relationships. 

With respect to the definition of subsystems and the design 
phase, categories of variables consistent with the elements of the 
Center's model of evaluation may be thought of as subsystems. Thus, 
a major task here is the identification of important variables within 
each category, development or adaptation of appropriate measures, 
identification of relationships among these variables and working 
out procedures for overcoming technological or methodological problems 
including sampling problems, multiple criterion problems, and mulit- 
variate analysis techniques. In addition, there are some of the more 
practical problems involving development of procedures for collecting 
data and data processing, preparation of computer programs, etc. 

With respect to testing of the system and the model, it is 
contemplated that both simulation testing and real-world testing will 
be required. Simulation testing may be useful in resolving sampling 
problems, technological problems, and procedural problems. Real- 
world testing is necessary for determining relationships among vari- 
ables, practical problems of data collection, applicability of models. 
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and the like. 

A second project deals with the development of a series of 
mathematical regression models. The development of these models 
constitutes an exploratory study to define relationships between 
inputs and outputs. Once these relationships have been defined 
and understood, a mathematical model will be derived for simulating 
the characteristics of secondary schools and explaining the effects 
of the school's characteristics upon its performance outputs. 

The study deals with an examination of approximately 400 
California 4-year high schools that have undergone the accreditation 
process through the Western Association of schools and colleges in 
the last several years. Data were obtained from this source and 
supplemented in small part by other readily available data items. 
From these data, 103 study variables were generated for use in the 
analysis. These were categorized into the elements of the concep- 
tual model that we have used to guide our efforts. 

In terms of this model all of the data items were classified 
as either being fixed inputs to the system, process characteristics 
of the system which are administratively controllable, or outcome 
measures. By fixed inputs, we referred to measures of the student 
entering the system as well as financial resources being provided to 
the system. (Measures of external systems, i.e., community charac- 
teristics both in terms of social, political and economic character- 
istics, were also considered as a part of the fixed inputs to the 
system.) 



HF 






,. vdM^m ..^ •«» /i: i i - - ^.^JumJii j. 



By process characteristics of the system we referred to the alter- 
natives for the utilization of the financial and human resources. In this 



category were included variables which dealt with line item budget allo- 
cations within the system, teacher-student ratios, teacher salary ranges, 
listings of traditional course sequences normally taken by selected groups 
of students, and other such variables which were expressions of the educa- 
tional policy of the school. The alternative processes that we worked with 
in this stage are not discrete processes, but are instead highly interre- 
lated characteristics which, in aggregate, may be thought of as sets of al- 
ternative processes. Lack of a program budgeting system in the schools and 
the data on specific educational programs that might be provided by such a 
system, makes it infeasible at this time to talk about program alternatives 
for this system in terms other than that indicated. 

The output measures that were used in the study were fairly standard 
kinds of variables which might have easily been improved upon. In the first 
iteration of this study we have assumed that the criterion dimensions in- 
cluded in the accreditation report are the measures of effectiveness for 
the implied system objectives, and are attempting to build a model based 
on these system objectives. The variables included which fell into this 
category were measures of student achievement on standardized tests of 
reading and mathematics (Qj, median, Q^), percentage of senior students 
entering junior college, state colleges, state universities, or private in- 
stitutions, and college grade point average of the student groups at each 
of the first three types of institutions. In addition, there were measures 
of the drop-out rate. Also, there were various measures designed to 



/ ; .. 

if 





















8 



show changes in student career aspirations over the course of the high 
school period as well as changes in student plans toward entering 
college. In all, 103 study variables were generated from the data. 

While this early stage of exploring the data base does not ordin- 
arily lead to any great system insights or provide opportunities, yet, 
for full scale systems analyses, there are some kinds of results that 
come out of this first stage of systems analysis. The preliminary 
analyses of interrelationships as well as the work of “muddying our 
hands" with the data have already led to feedback which is modifying 
the data source and which in future iterations will lead to the 
ability to perform more precise systems analyses. This has taken 
place in two ways: (1) through a modification of existing data items 
(such as making certain linear transformations of data, inspired 
by the results of single predictor non-linear regression analyses); 
and (2) through improved specification of objectives. 

This first phase of analysis which is primarily concerned with 
the examination of “orienting" techniques has provided some insights 
into present data items which are simply not appropriate as indica- 
tors either by virtue of their high interrelationship to other system 
characteristics which seem considerably more powerful as variables 
or by virtue of methodological difficulties in utilizing the data 
item as it presently exists. Furthermore, knowledge of other empir- 
ical research as well as our own intuition leads us to believe that 
certain existing characteristics of the system, primarily fixed inputs, 
are not adequately represented by appropriate variables. That is, 



Intuitively one can recognize certain characteristics of student In- 
puts which just simply have to be Included Into any systems study. 

In addition. It appears that certain process characteristics are not 
yet well defined. This would be analogous to talking about the precise 
description of the nature of the instructional treatment when the thing 
being evaluated Is a specific Instructional program. Here, we are talk- 
ing about an entire school as the system and It Is necessary to define 
precisely the nature of the administrative-teacher-student configuration 
that Is the process for that school. This Is no easy task, but the 
analysis of data and continuing Iteration help to point out areas where 
the process characteristics are not yet well defined and perhaps to 
suggest manners In which this definition might be Improved. Indeed, a 
systems study often leads to the selection or derivation of new processes. 

A second way In which this early stage of the analysis has helped 
to modify the system Is through the specification of Implied objectives. 
The tentative specification of these objectives leads In turn to the 
reconsideration and more precise definition of systems objectives In 
conjunction with those In administrative positions within the system. 
Learning about outputs and their relationships to objectives are 
some of the chief outputs of system studies. We should look at our 
outputs and objectives as carefully as we look at our model and Its 
inputs. If we begin with tentative objectives such as we did In 
this study, we should expect to replace and modify as we move along. 

It Is unlikely that we will be able to define satisfactory objectives 
at the beginning of a study. Analysis of existing educational systems. 
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unlike less complex instructional programs, very frequently cannot 
start with specifically pre-stated behavioral objectives. In such 
educational systems the range of possible objectives and educational 
consequences simply is not known. 

The third project to be discussed is of a somewhat different 
type. The first two discussed dealt with relatively complex systems, 
and outcomes dimensions that were quite broad. This project is much 
more limited in scope.* One of our colleagues at UCLA has attempted 

to utilize systems analytic techniques to develop a salary scheme which 

* 

effectively reflects the established priorities and objectives of 
educational systems and which overcomes some of the limitations of the 
commonly used fixed step salary schedule. The proposed scheme utilized 
linear programming techniques to develop a logical, internally consis- 
tent salary schedule for school district personnel. In this study a 
linear programming model for a school district salary structure was 
derived, then applied to an educational system. The ability of this 
approach to consider school district priorities in a logical and rational 
manner was demonstrated by solving the model for the maximization of 
beginning teacher salaries. This criterion of effectiveness would 
reflect a situation in which a school district established as a prior- 
ity or objective, the attraction of new young teachers. 

The model itself seeks to evaluate school district salaries accord- 
ing to nine dimensions. In addition to the amount of school district 
resources available, the salary structure is considered in the final 
determination of salaries. In short, the model selects a single, unique 
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salary structure which satisfies both the budgetary and hlerarchlal 

constraints which are Imposed on a system. The flexibility of using 

this method can be demonstrated by solving the model under various \ 

objective functions, such as paying higher salaries to ghetto area 

school district personnel. 

We hope that these three examples just presented help to pro- 
vide insights into the range of educational research problems which 

j 

might beneficially be examined from the systems analytic viewpoint. 
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